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What is claimed is: 

1 . A method for comprehensively representing video information in^ 
manner facilitating indexing of the video information, comprising tWstep of: 

5 dividing a continuous video stream into a plurality of video^cenes, each 

of said video scenes comprising one or more video frames including one key 
frame; and 

further comprising at least one of the steps of: 

dividing, using intra-scene motion analysi;?^^ at least one of said 
10 plurality of scenes into one or more layers; 

representing, as a mosaic, at least on^ of said pluraliy of scenes; 
computing, for at least one layer oi^cene, one or more content- 
related appearance attributes; and 

storing, in a database, said conjtfent-related appearance 
15 attributes or said mosaic representationsy 

2. The method of claim 1, wherein selected scenes are divided into a 
backgroimd layer and a foregroimd Im^er, and a mosaic representation of said 
selected scenes comprises a two dimensional mosaic representation. 



20 
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3, The method of claim 1, further comprising the steps of 
storing said pluralitj^of scenes in a mass storage imit; and 
retrieving, in respcmse to a database query, scenes associated with one 

or more desired attribut^ defined in said database query. 

4. The method of claim 3, wherein said mosaic representation comprises 
one of a two dimepsional mosaic, a three dimensional mosaic and a network of 
mosaics 



30 5. The mifethod if claim 1, wherein said step of computing a content-based 
appearan<^ attribute for a layer of a scene comprises the steps of: 
Mierating an image pyramid of said layer; 
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filtering, using one or more filters as^pcfated with said content-based 
appearance attribute, each subband o^^md image pyramid to produce 
respective one or more feature ma|Js associated with each subband; and 

integrating said one oirjs^re featia-e maps associated with each 
5 respective subband to pmduce respective attribute pyramid subbands, 

wherein each of said ataibute pyramid subbands comprises a content-based 
appearance attrijinite subband associated with a corresponding image 
pyramid subbed. 

10 6. The method of claim 5, wherein said content-based appearance 
attribute comprises at least one of a luminance attribute, a chrominance 
attribute and a texture attribute. 



7. The method of claim 5, wherein said step of filtering further comprises 
15 the step of: 

rectifying each of said one or more feature maps associated with each 
subband. 



8. The method of claim 5, further comprising the step of: 

20 collapsing said attribute pyramid subbands to produce a content-based 

appearance attribute. 

9. The method of claim 1, fiirther comprising the step of: 
receiving a request for video i^ormation substantially matching a 

25 desired content-related appearance attribute; and 

retrieving video fi-ames or scenes having at least one layer associated 
with content-related appearayfce attributes substantially matching said 
desired content-related appearance attribute. 

30 10. The method of claim 9, wherein said step of receiving a request 
comprises the steps of: 

identifying a query type and a query specification, said query type 
comprising one of a luminance, chrominance and texture query type, said 
query specification defining a desired property of said identified query type; 
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selecting a predetermined filter type associated with said identified 
query type; and 

calculating, using said predetermined filter type and said desired 
property, a desired content-related appearance attribute, said desired 
5 content-related appearance attribute being suitable for comparing to said 
content-related appearance attributes stored in said database. 

r 

11. The method of claim 1, further comprising the steps of: 

storing, in a database, ancillary information associated with one or 
10 more layers or fi-ames of one or more scenes. 




12. A method for generating a video informatiprfaatabase, comprising the 
steps of: 

segmenting a vid^o in&rmation.s^eam into a plurality of scenes; 
15 analyzing each off said pkir^JIxy of scenes to identify a plurality of 

attributes, including co^te/itpbased attributes, associated with each scene; 
and 

storing, in a dja^hsLse, said plurality of identified attributes associated 
with each scene- 

13. The method of claim 12, wherein said step of segmentij^g comprises the 
steps of: 

generating a descriptor vector of a predetermip^ type for each video 
frame of a video information stream; 
25 calculating a difference between descijfJior vectors of successive 

frames; and 

generating a scene cut indicium/ifi response to said calculated 
difference exceeding a threshold lev^E 



30 14. The method of claim 1^ wherein said step of segmenting comprises the 
steps of: 

generating, in a/first pass, a descriptor vector of a predetermined type 
for each video frame<of a video information stream; 
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calculating, using said generated de^riptor vectors, a descriptor vector 
threshold level; 

calculating, in a second passyd^ difference between descriptor vectors of 
successive frames; and 

generating a scene cutifidicium in response to said calculated 
difference exceeding a thre^old level. 



15. The method of claim 12, further comprising the stejfs of: 
identifying a key frame within each video segme^; and 

10 representing said plurality of scenes as two-c^mensional mosaics, 

three-dimensional mosaics or three-dimensionajmructures. 

16. The method of clai oa IB, wherein smd step of analyzing each of said 
plurality of scenes compiusesVthe steps of: 

15 filtering, using one or more-^ters associated with a first predetermined 

image attribute, at least onba-^)resentative frame of each scene to generate 
respective feature vectors o^aid first predetermined image attribute for 
each scene; and 

filtering, using^e or more filters associated with a second 

20 predetermined image attribute, said at least one representative frame of each 
scene to generate respective feature vectors of said second predetermined 
image attribxife for each scene. 
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^7, A method for browsing a video program stored in a mass storage imit, 
said video program comprising a plurality of video scenes, said video scenes 
comprising at least a representative videogame, said method comprising the 
steps of: 

providing a database associate with the stored video program, said 
database comprising attribute inrormation associated with each of said 
representative video frames; 

formulating a query^wilizing attribute information associated with a 
desired video frame; 

searching said/database to identify video frames substantially 
satisfying said qu^y; and 
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►m said mass storage unit, one or more of said identified 

18. The method of claim 17, wherein said step of formulating a query 
comprises the steps of: 

selecting a query type; 

selecting a query specification; and 

computing a multi-dimensional feature vector using said query type 
and query specification. 

19. The method of claim 18, wherein said query specification is selected by 
identifying a portion of a displayed image, and said multi-dimensional feature 
vector is calculated using said query type and said identified potion of said 
displayed image. 




retrieving. 



video fi:'amj 




20. The method of claim 19, further comprising the steps of: 
formatting, for subsequent presentation on a display device, each 

scene including one or more of said identified video firames; and 
transmitting said formatted scenes. 

21. A computer-readable medium having stored thereon a pluraM^ of 
instructions, the plurality of instructions including instructionspMmich, 
when executed by a processor, cause the processor to perfoim the steps of: 

(a) dividing a continuous video stream into a plup^ity of video scenes, 
each of said video scenes comprising one or more vykeo fi^ames including one 
key fi:'ame; and at least one of the steps of: 

(b) dividing, using intra-scene^otion analysis, at least one of 
said plurality of scenes into otie or more layers; 

(c) representing, asa^osaic, at least one of said pluraliy of 

scenes; 

(d) computm^ at least one layer or scene, one or more content- 
related appearance ajtnbutes; and 

(e) storing, in a database, said content-related appearance 
attributes or s^rfd mosaic representations. 
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